Skip to content

Add scalar function performance baselines - #9136

Open
connortsui20 wants to merge 1 commit into
ct/l2-denorm-encodingfrom
ct/scalar-fn-baselines
Open

Add scalar function performance baselines#9136
connortsui20 wants to merge 1 commit into
ct/l2-denorm-encodingfrom
ct/scalar-fn-baselines

Conversation

@connortsui20

@connortsui20 connortsui20 commented Aug 1, 2026

Copy link
Copy Markdown
Member

Summary

Tracking Issue: #9128

Adds stable scalar function performance baselines for byte_length, l2_norm, normalized, inner_product, cosine_similarity, GeoContains, and GeoIntersects.

The benchmark sizes are all comfortably above the roughly 146-instruction Divan harness floor from #9011. Each benchmark binary also uses vendored mimalloc, since scalar function execution allocates its output inside the timed trace and glibc differences between runner images caused the flakes fixed in #8861.

Why

Codspeed can only compare a later implementation change when the same benchmark name already exists on develop, so these baselines need to land before the RowFn things I want to make. And these are pretty cheap, might as well add them.

@codspeed-hq

codspeed-hq Bot commented Aug 1, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 1885 untouched benchmarks
🆕 75 new benchmarks
⏩ 1 skipped benchmark1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation inline[4096] N/A 61.8 µs N/A
🆕 Simulation inline[65536] N/A 787.5 µs N/A
🆕 Simulation nullable_out_of_line[4096] N/A 68.2 µs N/A
🆕 Simulation nullable_out_of_line[65536] N/A 793.9 µs N/A
🆕 Simulation out_of_line[4096] N/A 62.1 µs N/A
🆕 Simulation out_of_line[65536] N/A 787.7 µs N/A
🆕 Simulation add_i32_nonnull N/A 416.4 µs N/A
🆕 Simulation add_i64_constant N/A 567.1 µs N/A
🆕 Simulation add_u32_nonnull N/A 416.3 µs N/A
🆕 Simulation nullable_out_of_line_90pct[4096] N/A 73.9 µs N/A
🆕 Simulation nullable_out_of_line_90pct[65536] N/A 794 µs N/A
🆕 Simulation like_per_row_distinct_patterns N/A 1.1 ms N/A
🆕 Simulation like_per_row_repeated_patterns N/A 242.1 µs N/A
🆕 Simulation column_x_column_points N/A 14.3 ms N/A
🆕 Simulation column_x_column_polygons N/A 23.8 ms N/A
🆕 Simulation constant_x_nullable_points N/A 9.1 ms N/A
🆕 Simulation constant_x_nullable_polygons_disjoint N/A 12.3 ms N/A
🆕 Simulation constant_x_points N/A 9.6 ms N/A
🆕 Simulation constant_x_polygons_disjoint N/A 11.4 ms N/A
🆕 Simulation constant_x_polygons_overlapping N/A 123.4 ms N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing ct/scalar-fn-baselines (9755708) with ct/l2-denorm-encoding (c2288dc)

Open in CodSpeed

Footnotes

  1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports.

@joseph-isaacs

Copy link
Copy Markdown
Contributor

This benchmark has a too long runtime

@connortsui20
connortsui20 force-pushed the ct/scalar-fn-baselines branch from 880ddac to 69edd3a Compare August 3, 2026 15:57
@connortsui20
connortsui20 force-pushed the ct/scalar-fn-baselines branch 2 times, most recently from f599518 to 9fcb4a7 Compare August 3, 2026 19:56
@connortsui20
connortsui20 force-pushed the ct/scalar-fn-baselines branch 2 times, most recently from 1dccaaf to bf814bb Compare August 3, 2026 20:30
@connortsui20
connortsui20 changed the base branch from ct/scalar-fn-factory-ext to ct/l2-denorm-encoding August 3, 2026 20:37
@connortsui20
connortsui20 force-pushed the ct/scalar-fn-baselines branch from bf814bb to 003cc85 Compare August 3, 2026 21:19
@connortsui20 connortsui20 added the changelog/performance A performance improvement label Aug 3, 2026
Adds stable CodSpeed baselines for public construction and execution paths across byte length, binary arithmetic, LIKE, tensor scalar functions, and geo operations. The benches avoid RowFn test-harness internals so later implementation changes compare against the production path.

`like_per_row_patterns` keeps its original repeated `hello%` fixture so its CodSpeed history stays comparable. The compile-cache pair lands under new names: `like_per_row_repeated_patterns` and `like_per_row_distinct_patterns` share a five-byte pattern shape and differ only in whether the cache hits.

Geo distance covers point and polygon columns, including constants and nullable points. Geo predicate coverage adds sparse and mixed validity cases, while its all-overlapping polygon arm uses 1,024 rows because CodSpeed simulates the compiled amd64/AVX2 machine code.

Each benchmark binary that allocates inside the timed region uses vendored `mimalloc`. `Normalized` keeps its checked inherent constructor because its normalized-row invariant requires construction-time validation.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 force-pushed the ct/scalar-fn-baselines branch from 003cc85 to 9755708 Compare August 3, 2026 21:40
@connortsui20
connortsui20 marked this pull request as ready for review August 3, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants